To overcome the shortcoming that the Iterative Closest Point (ICP) algorithm is easy to fall into local optimum, an auto-registration method of ground based building point clouds based on line features and ICP algorithm was proposed. Firstly, the plane segmentation was performed on point clouds based on normal consistency. Secondly, the outlines of point clusters were extracted by alpha-shape algorithm, and the feature line segments were obtained by the splitting and fitting process. Then, the feature line pairs were taken as the registration primitives, and the angle and distance between line pairs were used as similarity measures for same-name feature matching in order to achieve the coarse registration of building cloud points. Finally, with the coarse registration result as the initial value, the ICP algorithm was used to realize the fine registration of building point clouds. Two sets of partially overlapping building point clouds were used to carry out the experiments. The experimental results show that the proposed coarse-to-fine registration method can effectively improve the dependency of ICP algorithm on initial value and realize the effective registration of partially overlapping building point clouds.